Smaller generalization error derived for a deep residual neural network compared with shallow networks

Abstract Estimates of the generalization error are proved for a residual neural network with $L$ random Fourier features layers $\bar z_{\ell +1}=\bar z_\ell + \textrm {Re}\sum _{k=1}^K\bar b_{\ell k}\,e^{\textrm {i}\omega _{\ell k}\bar }+ c_{\ell ^{\prime}_{\ell k}\cdot x}$. An optimal distribution frequencies $(\omega k},\omega k})$ $e^{\textrm }$ and x}$ is derived. This derivation based on corresponding approximation function values $f(x)$. The turns out to be smaller than estimate ${\|\hat f\|^2_{L^1({\mathbb {R}}^d)}}/{(KL)}$ features, one hidden layer same total number nodes $KL$, in case $L^\infty $-norm $f$ much less $L^1$-norm its transform $\hat f$. understanding an used construct new training method deep network. Promising performance proposed algorithm demonstrated computational experiments.